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In  1990,  Congress  passed  section  831  of  the  National  Defense 
Authorization  Act  for  Fiscal  Year  1991  establishing  the  Pilot 
Mentor-Protege  Program.  The  purpose  of  the  program  is  to  provide 
incentives  for  major  Department  of  Defense  (dod)  contractors  (mentors) 
to  furnish  disadvantaged  small  business  concerns^  (proteges)  with 
assistance  designed  to  enhance  their  capabilities  and  increase  their 
participation  as  subcontractors  and  suppliers  under  dod  contracts,  other 
federal  government  contracts,  and  commercial  contracts. 

The  fiscal  year  1998  National  Defense  Authorization  Act  requires  us  to 
report  on  dod’s  implementation  of  the  program  and  the  extent  to  which 
the  program  is  achieving  the  purposes  established  by  Congress.  We 
reviewed  (1)  dod’s  management  of  the  program,  including  its  efforts  to 
evaluate  the  program’s  effectiveness  and  (2)  the  manner  in  which  program 
funds  have  been  obligated.  We  also  attempted  to  assess  the  extent  to 
which  the  pilot  program  benefited  the  proteges. 


Background 


dod’s  Office  of  Small  and  Disadvantaged  Business  Utilization  (osadbu)  is 
responsible  for  the  mentor-protege  program.  A  signed  mentor-protege 
agreement  for  each  mentor-protege  relationship  must  be  submitted  to 
OSADBU  and  approved  before  developmental  assistance  costs  may  be 
incurred,  osadbu  has  reimbursed  mentors  through  cooperative  agreements 
where  both  the  government  and  a  contractor  work  together  to  achieve  a 


^To  qualify  as  a  disadvantaged  small  business,  a  company  must  not  exceed  the  Small  Business 
Administration’s  standards  for  number  of  employees  or  annual  sales  and  must  be  independently 
owned  (at  least  61  percent)  and  operated  by  socially  and  economically  disadvantaged  individuals. 
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common  purpose,  osadbu  also  approved  credit  agreements  that  were 
negotiated  on  a  nonreimbursable  basis.^  In  addition,  the  services  and  the 
defense  agencies  requested  osadbu’s  approval  of  mentor-protege 
agreements  that  reimbursed  mentors  pursuant  to  separate  contracts  or 
line  items  in  dod  contracts,  osadbu  is  to  conduct  periodic  performance 
reviews  of  the  progress  and  accomplishments  realized  under  approved 
mentor-protege  agreements. 

This  report  is  one  of  a  series  we  have  issued  on  the  mentor-protege 
program.  In  our  first  report,  we  recommended,  among  other  things,  that 
DOD  develop  and  implement  adequate  internal  controls  in  the  application 
and  approval  process  and  in  the  oversight  of  protege  development.®  In  our 
second  report,  we  were  not  able  to  recommend  that  the  pilot  program  be 
extended  because  sufficient  information  was  not  available  to  determine 
whether  the  program’s  purposes  could  be  achieved  or  whether 
reauthorization  and  extension  were  warranted.^ 


Results  in  Brief 


Data  limitations  preclude  our  assessing  the  extent  to  which  the  program  is 
achieving  the  purposes  established  by  Congress,  osadbu  has  recently 
undertaken  actions  to  review  the  program  that  were  intended  to  provide 
the  basis  for  such  an  assessment.  It  has  initiated  a  survey  of  mentors  and 
proteges  and  enlisted  the  Defense  Contract  Management  Command  to 
conduct  performance  evaluations  of  each  agreement.  However,  we  believe 
shortcomings  in  the  survey  methodology  and  incomplete  coverage  in  the 
performance  evaluations  will  limit  the  usefulness  of  their  results  in 
assessing  the  program’s  overall  effectiveness. 

Congress  has  appropriated  about  $233  million  for  the  program  since  fiscal 
year  1992.  The  fimding  was  generally  obligated  through  either  cooperative 
agreements,  separate  contracts,  or  line  items  in  dod  prime  contracts.  While 
osadbu  has  managed  cooperative  agreements  in  the  past,  it  has  decided 
that  the  services  and  the  defense  agencies  should  be  responsible  for 
managing  reimbursable  mentor-protege  agreements.  In  addition,  the 
services  have  been  inconsistent  in  their  policies  on  paying  fees  to  mentors 


^By  using  a  credit  agreement,  a  mentor  is  reimbursed  for  developmental  assistance  costs  that  are 
allowable,  allocable,  and  reasonable  through  their  inclusion  in  indirect  cost  pools  and  is  also  given 
credit  toward  meeting  small  business  contracting  goals  as  if  these  assistance  costs  were  contracts 
awarded  to  small  and  disadvantaged  businesses. 

^Defense  Contracting:  Interim  Report  on  Mentor-Protege  Program  for  Small  Disadvantaged  Firms 
(G.AO/NBlADsl2-:3::>,  Mar.  30,  1992)r  ” 

^Defense  Contracting:  Implementation  of  the  Pilot  Mentor-Protege  Program  (GAO^'NSIAD-fri-lO  1 , 
Feb.  1, 1994). 
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for  providing  assistance  to  proteges  and  reimbursing  proteges  for  various 
expenses. 

Because  sufficient  and  reliable  information  on  program  performance  was 
imavailable,  the  overall  extent  to  which  the  pilot  program  benefited  the 
proteges  could  not  be  assessed. 


Management  Actions 
Are  Underway 


In  our  prior  two  reports,  we  identified  the  lack  of  management  data, 
oversight  of  individual  agreements,  and  evaluation  of  results  of  program 
participation  as  major  impediments  to  measuring  progress  toward 
accomplishing  congressional  goals.  Within  the  last  16  months,  osadbu  has 
initiated  actions  to  evaluate  program  performance  and  improve 
administration  of  the  program.®  Actions  to  evaluate  program  performance 
include  conducting  a  survey  of  program  participants  and  establishing  a 
performance  review  process.  However,  low  survey  response  rates  and 
incomplete  performance  review  data  weaken  osadbu’s  efforts  to  evaluate 
and  report  on  the  overall  success  of  the  program. 


Our  Prior  Reviews  of  the 
Program 


Our  1992  report  pointed  out  that  dod  needed  to  develop  and  implement 
adequate  internal  controls  in  the  application  and  approval  process  and  in 
the  oversight  of  protege  development.  In  its  response,  osadbu  officials 
agreed  to  verify  the  progress  reports,  ensure  the  agreed  upon  assistance 
was  provided,  and  evaluate  (through  communication  with  the  proteges) 
whether  the  assistance  had  increased  the  number  of  subcontracts 
awarded. 

In  our  1994  report,  we  again  pointed  out  that  osadbu  had  neither 
(1)  assessed  progress  and  accomplishments  realized  under  any  of  the 
agreements  nor  (2)  evaluated  program  success.  In  its  comments  to  our 
report,  osadbu  acknowledged  that  monitoring  and  oversight  of 
mentor-protege  agreements  had  not  occurred  and  added  that  it  had  plans 
to  do  so  once  resources  were  available  and  enough  time  had  elapsed  for 
significant  activity  under  the  agreements  to  have  occurred. 


“OSADBU  officials  told  us  that  actions  have  been  taken  to  improve  program  administration,  including 
tracking  financial  information,  abandoning  centrally  managed  cooperative  agreements,  moving  toward 
service-funded  agreements,  implementing  a  phased  approach  to  reduce  cost  of  agreements,  increasing 
the  use  of  incrementally  funding  of  agreements,  and  improving  outreach  briefings  to  provide  clear 
guidance  and  lessons  learned  to  participants. 
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OSADBU  Has  Initiated 
Action 


Survey  Provides  Limited  Data 
on  Program’s  Effectiveness 


In  November  1996,  osadbu,  with  contractor  support,  began  a  data 
collection  effort  by  conducting  a  survey  of  program  participants  to 
evaluate  program  effectiveness.  Also,  osadbu  requested  that  the  Defense 
Contract  Management  Conunand  (dcmc)  conduct  performance  reviews  of 
all  mentor-protege  agreements  approved  since  the  program’s  inception, 
and  thereafter,  to  review  all  active  agreements  annually.  The  purpose  of 
these  reviews  is  to  establish  and  maintain  a  regular  performance  review 
process. 

During  our  review,  osadbu  was  conducting  a  questionnaire  survey  of  the 
program,  osadbu  officials  told  us  that  the  questionnaire  responses  will,  in 
part,  provide  the  data  needed  to  ascertain  and  report  overall  progress 
toward  achieving  the  program’s  statutory  goals,  osadbu  plans  to  issue  its 
report  by  late  spring  1998. 

Separate  questionnaires  for  mentors  and  proteges  were  sent  to  current 
participants  as  well  as  those  who  had  completed  or  terminated  agreements 
as  of  November  4, 1996.  The  questionnaires  requested  general  background 
data  on  the  participants  and  the  agreements,  information  on  types  of 
assistance  provided,  data  on  the  dollar  value  and  number  of  contracts  as 
well  as  subcontracts  received  and/or  awarded  during  the  agreements,  and 
experiences  of  the  participants  and  their  views  of  the  program. 

A  successful  survey  relies  on  a  high  response  rate,  an  analysis  of  why 
nonrespondents  in  the  population  did  not  respond,  and  an  assessment  of 
the  validity  and  reliability  of  the  data.  As  of  January  1998,  osadbu’s 
response  rates  of  about  60  percent  for  the  mentor  survey  and  about 
40  percent  for  the  protege  survey  were  well  below  that  which  would  be 
required  to  draw  statistically  meaningful  conclusions  about  the  population 
of  program  participants  or  the  mentor-protege  program  as  a  whole.® 
Further,  osadbu  has  not  conducted  a  systematic  analysis  of  why 
nonrespondents  did  not  participate  in  the  survey  to  assess  any  patterns  for 
their  lack  of  participation  as  compared  to  the  respondents. 

Also,  the  financial  data  on  contracting  and  subcontracting  were  not 
systematically  verified  with  an  independent  source  such  as  progress 
reports,  contractor  reports,  or  other  records.  Attempts  were  made  to 
reconcile  conflicting  information  between  the  mentors  and  proteges,  but 
only  if  the  discrepancies  were  large  or  if  totally  inconsistent  information 
was  provided  by  each  respondent. 


^’To  make  plausible  generalizations  about  the  overall  pilot  program,  the  effective  response  rate  should 
usually  be  at  least  76  percent  for  each  variable  measure — a  goal  used  by  most  practitioners.  See 
Developing  and  Using  Questionnaires  (GAO/PEMD-10.1.7,  Oct.  1993). 
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Performance  Evaluations  Have  On  November  6, 1996,  osadbu,  in  an  effort  to  conduct  annual  performance 

Shortcomings  reviews  of  all  mentor-protege  agreements,  requested  dcmc  to  review  all 

305  agreements  that  had  been  approved  since  the  program’s  inception.  In 
response,  dcmc  sent  a  memorandum  dated  January  2, 1997,  to  its  districts 
setting  out  its  policy  on  conducting  the  performance  reviews.  The  reviews 
were  to  include  evaluations  of  costs,  planned  and  actual  mentor 
assistance,  and  data  on  protege  firms’  business  development  (contract 
awards  and  improvements  in  technical  capabilities)  resulting  from 
program  participation.  Although  dcmc  requested  that  all  the  reports  be 
sent  to  OSADBU  by  December  31, 1997,  osadbu  had  received  only  138 
reports  on  the  305  agreements,  or  about  45  percent,  as  of  that  date. 

We  reviewed  66  of  the  138  dcmc  reports  and  found  that  many  of  the  reports 
were  missing  information  or  did  not  address  all  the  required  areas  outlined 
in  the  dcmc  policy  memorandum.  The  differences  in  reporting  all 
information,  as  well  as  incomplete  data,  will  limit  the  usefulness  of  the 
performance  reviews  in  analyzing  trends,  tracking  performance  of 
program  agreements,  and  assessing  the  program’s  effectiveness,  osadbu’s 
support  contractor  is  reviewing  the  reports  to  identify  those  that  are 
missing  and  the  general  quality  and  consistency  of  the  reviews  to 
determine  what  additional  dcmc  guidance  is  necessary  for  its  districts. 


Manner  of  Obligating 
Funds  Has  Changed 


The  funding  appropriated  from  fiscal  year  1992  to  1998  was  generally 
obligated  through  either  cooperative  agreements,  separate  contracts,  or  in 
a  line  item  in  dod  prime  contracts.  While  osadbu  has  managed  cooperative 
agreements  in  the  past,  it  has  decided  that  the  services  and  the  defense 
agencies  should  be  responsible  for  managing  reimbursable  mentor-protege 
agreements.  In  addition,  the  services  were  inconsistent  in  paying  fees  to 
mentors  and  reimbursing  proteges  for  various  expenses. 


Changes  in  the  Ways  Funds 
Are  Obligated 


Mentor-protege  funding  of  about  $233  million’’  was  generally  obligated 
through  cooperative  agreements,  separate  contracts,  or  line  items  in  dod 
contracts,  osadbu  spent  about  $88.3  million  on  50  cooperative  agreements 
that  resulted  in  99  mentor-protege  agreements  ranging  in  value  from 
$135,000  to  about  $3  million,  or  an  average  of  about  $892,000  per 
mentor-protege  agreement.  In  addition,  the  services  and  the  defense 


’^In  addition  to  the  $233  million,  an  undetermined  amount  was  believed  to  be  reimbursed  indirectly 
through  142  credit  agreements  by  including  the  costs  of  mentor  assistance  in  indirect  costs  pools. 
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agencies  have  reimbursed  costs  amounting  to  about  $108.6  million,® 
pursuant  to  143  separate  contracts  or  line  items  in  dod  contracts  that  have 
ranged  in  value  from  $38,525  to  $6.9  million,  or  an  average  of  about 
$769,000  per  mentor-protege  agreement.  Funds  amounting  to  about 
$7.2  million  were  also  disbursed  by  osadbu  for  historically  Black  colleges 
and  universities  and  minority  institutions,  contractor  support,  and  other 
miscellaneous  expenses.  As  of  February  1998,  the  remaining  funds  of 
$28.9  million  included  $10.6  million  that  had  been  sent  to  the  services  but 
not  yet  obligated,  $16.6  million  that  had  not  been  distributed,®  and 
$1.7  million  that  had  expired. 

According  to  the  osadbu  program  manager,  centrally  managed  cooperative 
agreements  were  useful  for  initially  getting  the  program  started  because 
they  successfully  produced  interest  in  the  program,  osadbu  used  a 
“program  announcement”  whereby  potential  mentors  submitted  proposals 
for  funding  a  mentor-protege  agreement  through  a  cooperative  agreement. 

OSADBU  has  decided  to  discontinue  its  use  of  centrally  managed 
cooperative  agreements.  The  osadbu  program  manager  informed  us  that 
the  level  and  type  of  technical  assistance  provided  to  the  proteges  were 
primarily  business  in  nature,  and  not  related  to  DOD-specific  program 
areas.  The  program  manager  further  pointed  out  that  osadbu  does  not  have 
the  staff  to  centrally  manage,  review,  monitor,  and  serve  as  the  responsible 
official  on  multiple  cooperative  agreements. 

osadbu  has  decided  that  the  services  and  the  defense  agencies  should  be 
responsible  for  managing  reimbursable  mentor-protege  agreements.  The 
osadbu  program  manager  informed  us  that  by  linking  the  agreements  to  a 
service  or  a  defense  agency,  the  likelihood  of  a  protege  receiving  technical 
assistance  that  is  mission  related  increases  significantly.  Further,  by 
incorporating  the  program  into  the  acquisition  process,  it  becomes  an 
integral  part  of  that  process,  thus  mitigating  the  need  for  additional 
resources  to  administer  the  program  and  monitor  program  performance.^ 


®Iii  addition  to  the  $108.5  million,  Air  Force  funding  of  $1 1.6  million  was  used  to  support  four 
agreements  developing  technologies  for  the  F-22  aircraft,  while  Navy  funding  of  $900,000  was  used  to 
support  three  agreements  for  the  F/A-18  aircraft. 

^Included  in  this  amount  was  $10  million  available  for  fiscal  year  1998,  but  withheld  by  the  DOD 
Comptroller. 

^•^Since  the  mentor-protege  agreements  are  incorporated  into  contracts,  the  service  OSADBU  program 
managers  generally  view  the  administration  of  the  agreements  to  be  a  function  of  the  contracting 
officers  within  the  program  offices. 
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Services  Are  Inconsistent 
in  Reimbursing  Various 
Expenses 


Extent  Proteges 
Benefited  From 
Funding  Has  Not  Been 
Determined 


Neither  the  mentor-protege  program  legislation  nor  the  implementing 
regulation  specifically  addresses  either  the  payment  of  fees  to  mentors  or 
the  reimbursement  of  proteges  for  their  costs  in  participating  in  the 
program. 

In  our  March  1992  report,  issued  shortly  after  the  program  actually  began, 
we  noted  low  mentor  participation  and  recommended  that  dod  consider 
ways  to  enhance  program  incentives  for  mentor  participation.  At  that  time, 
we  noted  that  the  appeal  of  the  cash  reimbursement  incentive  was 
diminished  because  mentors  were  prohibited  by  regulation  from  earning  a 
fee  on  the  development  assistance  provided  to  proteges.  In  1992,  dod 
eliminated  the  regulatory  prohibition. 

In  our  current  review,  we  found  that  the  payment  of  fees  to  mentors  varied 
among  the  services.  We  found  no  instance  where  the  Navy  has  paid  a  fee, 
while  the  Army  and  the  Air  Force  have  paid  fees  to  mentors.  According  to 
the  OSADBU  program  manager,  mentors  have  received  fees  in  addition  to 
their  expenses  because  mentor-protege  agreements  are  reimbursed  by 
means  of  a  line  item  in  dod  contracts  that  allow  a  fee  and  the  program’s 
legislation  does  not  prohibit  the  payment  of  fees  to  the  mentor. 

In  May  1997,  the  Air  Force  asked  osadbu  about  reimbursing  protege 
salaries  while  the  proteges  were  undergoing  training.  The  Air  Force  noted 
that  it  had  found  no  prohibition  on  such  payments  in  either  the  program 
legislation  or  the  regulation,  osadbu  approved  the  payment  of  such 
incidental  expenses  in  July  1997,  so  long  as  they  were  otherwise, 
allowable,  allocable,  and  reasonable  and  did  not  exceed  10  percent  of  the 
total  reimbursable  cost  of  the  approved  mentor-protege  agreement.  We 
found  that  the  Army  and  the  Air  Force  permit  these  payments;  the  Navy 
does  not. 


While  the  mentor-protege  program  has  provided  incentives  to  mentors  to 
provide  assistance  to  proteges,  the  extent  that  the  funding  has  enhanced 
the  business  competitiveness  and  financial  independence  of  protege  firms 
has  not  been  determined  because  sufficient  and  reliable  information  on 
program  performance  has  not  been  available,  osadbu  is  developing  a 
program  database  that  it  believes  will  be  the  source  for  periodic  analysis 
of  program  results. 

About  180  mentors  received  reimbursement  of  certain  costs  incurred  in 
providing  assistance  to  their  protege  firms,  credit  in  meeting  small 
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business  goals, “  or  both.  Of  the  180  mentors,  92  used  only  reimbursable 
agreements,  69  used  only  credit  agreements,  and  19  used  both 
reimbursement  and  credit  agreements.  We  found  that  86  mentors  have 
active  agreements,  while  94  mentors  have  completed  or  terminated  their 
agreements. 

Of  the  340  proteges  that  received  assistance  from  mentors,  201  received 
assistance  through  reimbursable  agreements,  113  received  assistance 
through  credit  agreements,  and  26  used  both  types  of  agreements.  We 
foxmd  that  147  proteges  have  active  agreements,  while  193  proteges  have 
completed  or  terminated  their  agreements. 


OSADBU  Is  Developing  a  OSADBU  is  developing  a  program  database  on  all  participants,  including 

Program  Database  responses  to  the  questionnaire  surveys.  The  database  is  expected  to 

incorporate  information  gathered  from  the  mentor  application,  the 
agreements,  and  the  data  gathered  as  a  result  of  the  dcmc  reviews.  In 
addition,  a  reference  resource  is  expected  to  provide  more  easily 
understood  program  guidelines  and  streamlined  procedures  for  program 
participation  and  guidance  on  costs  eligible  for  reimbursement,  lessons 
learned,  and  best  practices. 


Conclusion 


After  spending  over  $200  million  on  the  pilot  mentor-protege  program, 
OSADBU  does  not  know  the  overall  extent  to  which  the  program  is  achieving 
the  purposes  Congress  has  established  because  sufficient  and  reliable 
information  on  program  performance  is  unavailable.  While  osadbu  has 
initiated  an  assessment  of  the  program  by  conducting  a  survey  and 
establishing  a  performance  review  process,  shortcomings  in  the 
assessment  will  limit  its  usefulness.  For  example,  a  low  survey  response 
rate  will  limit  the  usefulness  of  its  findings  in  assessing  the  program’s 
overall  effectiveness.  In  addition,  performance  reviews  were  missing 
information  or  did  not  address  aU  the  required  areas,  such  as 
improvements  in  protege  capabilities  and  contract  awards.  While  it  would 
be  difficult  to  correct  the  shortcomings  in  the  survey  to  assess  the  overall 
effectiveness  of  the  program,  the  weaknesses  in  the  performance  reviews 
can  be  overcome. 

Neither  the  mentor-protege  program  legislation  nor  the  implementing 
regulation  specifically  addresses  either  the  payment  of  fees  to  mentors  or 


acquisitions  above  $600,000  ($1  million  for  construction),  prime  contractors  are  required  to 
establish  a  subcontracting  plan  that  specifies  the  percentage  goal  of  subcontract  awards  to  small  and 
disadvantaged  businesses. 


Page  8 


GAO/NSIAD- 98-92  DOD’s  Mentor-Protege  Program 


B-278784 


the  reimbursement  of  proteges  for  their  costs  in  participating  in  the 
program.  We  found  that  there  is  no  uniformity  among  the  services  on 
making  these  payments. 

Recommendation 

We  recommend  that  the  Secretary  of  Defense  instruct  the  Director, 

OSADBU,  to  strengthen  osadbu’s  performance  reviews  to  ensure  that 
sufficient  and  reUable  information  on  program  performance  is  gathered  on 
planned  and  actual  mentor  assistance  as  well  as  on  the  protege  firms’ 
business  development. 

Matter  for 

Congressional 

Consideration 

Congress  may  wish  to  clarify  mentor-protege  program  legislation  as  to 
whether  mentors  can  be  paid  fees  in  addition  to  expenses  and  proteges 
can  be  reimbursed  for  various  expenses. 

Agency  Comments 

DOD  concurs  with  the  recommendation  to  strengthen  performance  reviews 
and  will  continue  to  make  efforts  to  improve  the  reliability  and  sufficiency 
of  data  on  the  program.  However,  dod  states  that  there  have  been  a 
significant  number  of  proteges  that  have  benefited  from  the  program  and 
provides  nine  examples  of  mentor-protege  agreements  that  it  believes 
illustrate  the  effectiveness  of  the  program.  We  found  that  there  has  been  a 
lack  of  sufficient  and  reliable  information  on  close  to  400  agreements 
approved  by  osadbu  to  assess  the  program’s  overall  effectiveness  and  do 
not  believe  that  nine  examples,  while  positive  in  themselves,  are  a 
sufficient  basis  for  this  assessment.  (See  app.  I  for  dod’s  written 
comments.) 

DOD  acknowledges  that  the  services  have  been  inconsistent  in  their 
policies  on  paying  fees  to  mentors  for  providing  assistance  to  proteges  and 
reimbursing  proteges  for  various  expenses.  By  allowing  the  diversity  of 
approaches,  dod  states  that  it  is  testing  the  range  of  options  for  program 
execution  and  the  impact  of  various  incentives  upon  participation  in  the 
program.  Given  that  the  mentor-protege  program  legislation  and  the 
implementing  regulation  do  not  specifically  address  either  paying  fees  to 
mentors  or  reimbursing  proteges  for  their  costs  in  participating  in  the 
program,  we  continue  to  believe  that  Congress  may  wish  to  clarify  existing 
legislation  as  to  the  use  of  mentor-protege  funding. 

Page  9 


GAO/NSIAD-98-92  DOD’s  Mentor-Protege  Program 


B-278784 


Scope  and 
Methodology 


The  congressional  mandate  states,  in  part,  that  we  will  study  the  extent  to 
which  the  program  is  achieving  the  purposes  established  by  Congress. 
Because  dod  maintained  performance  review  and  evaluation  data^^  on  the 
program  that  was  insufficient  and  unreliable,  we  considered  making  an 
independent  data  collection  survey.  We  did  not  make  such  a  survey 
because  osadbu  and  a  trade  association  were  completing  separate  surveys 
of  the  same  population  of  mentors  and  proteges.  We  also  did  not  make  the 
survey  because  repeated  surveys  of  the  same  population  at  closely  spaced 
intervals  can  lead  to  “history  effects,”  which  can  effect  both  the  response 
rate  and  the  quality  of  the  data  collected  in  unpredictable  ways.  We 
focused  our  review  on  the  management  of  the  program. 

To  review  program  management,  we  reviewed  our  past  reports  on  the 
program  that  identified  weaknesses  in  the  management,  obtained  dod’s 
response  to  our  findings,  and  identified  osadbu’s  actions  initiated  to 
improve  program  administration  and  evaluation.  We  also  contacted 
osadbu’s  contractor  who  is  responsible  for  administering  and  summarizing 
the  DOD  survey  questionnaire  and  dcmc  to  identify  its  progress  in 
conducting  performance  reviews  of  the  mentor-protege  agreements. 

To  review  the  manner  in  which  the  funds  were  obligated,  we  met  with 
OSADBU  and  Washington  Headquarters  Service  personnel  who  transfer 
mentor-protege  funding  from  osadbu  to  the  responsible  services  and  the 
defense  agencies.  As  such,  we  obtained  overall  funding  information 
dealing  with  the  program  from  fiscal  year  1992  to  1998  that  showed  a 
breakdown  of  the  funding  by  responsible  organization.  To  verify  the 
funding  information,  we  contacted  officials  within  the  services’  and 
defense  agencies’  offices  of  Small  and  Disadvantaged  Business  Utilization 
responsible  for  administering  the  mentor-protege  agreements.  A^Tien 
necessary,  these  officials  contacted  supporting  commands  to  gather  more 
reliable  information  regarding  the  obligation  of  the  funding. 

We  attempted  to  assess  the  extent  to  which  the  funds  benefited  the 
proteges  by  reviewing  osadbu’s  files  and  contacting  service  and  defense 
agency  officials  to  identify  the  number  of  mentors  and  proteges  that  have 
participated  in  the  program.  Sufficient  and  reliable  information  on 
program  performance,  however,  was  imavailable  on  the  overall  extent  that 
the  proteges  benefited  from  the  funding  provided  by  the  program. 


^^According  to  OSADBU  officials,  the  data  were  useful  in  making  program  administrative  decisions 
such  as  abandoning  centrally  managed  cooperative  agreements  and  moving  toward  service-funded 
agreements. 
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We  performed  this  phase  of  work  between  August  1997  and  February  1998 
in  accordance  with  generally  accepted  government  auditing  standards. 


We  will  send  copies  to  the  Chairmen  and  the  Ranking  Minority  Members 
of  other  appropriate  congressional  committees;  the  Secretary  of  Defense; 
and  the  Director,  Office  of  Management  and  Budget.  We  will  also  make 
copies  available  to  others  upon  request. 

If  you  have  any  questions  regarding  this  report,  please  call  me  on 
(202)  5124841.  Major  contributors  to  this  report  are  listed  in  appendix  II. 


Louis  J.  Rodrigues 

Director,  Defense  Acquisitions  Issues 
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Comments  From  the  Department  of  Defense 


Note;  GAO  comments 
supplementing  those  in  the 
report  text  appear  at  the 
end  of  this  appendix. 


OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 

3000  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301*3000 

ACOUlSJTiON  AND 
TEC*^MOLOGY 

Februaty  26,  1 998 

Mr.  David  E.  Cooper 

Associate  Director,  Defense  Acquisitions  Issues 
National  Securit>'  and  International  Affairs  Division 
U,S.  General  Accounting  Office 
Washington,  DC  20548 

Dear  Mr.  Cooper: 

Thank  you  for  the  opportunity  to  comment  on  the  draft  GAO  report,  “Defense 
Contracting:  Sufficient  Reliable  Information  on  DoD’s  Mentor-Protege  Program  is 
Unavailable”  (GAO  Code  707292/OSD  Case  1547).  Before  getting  to  the  substance  of 
our  review  of  the  report,  please  accept  our  compliments  on  the  way  in  which  your 
personnel  worked  with  members  of  the  Department  in  compiling  and  preparing  this  report 
within  restrictive  time  constraints.  Your  staff  has  made  a  consistent  effort  to 
accommodate  our  concerns  while  answering  the  questions  they  have  been  asked  to  re\iew 
in  compliance  with  existing  GAO  guidelines. 

The  DoD  Pilot  Mentor-Protege  Program  (MPP)  is  a  unique  effon.  Through  this 
program  the  Department  seeks  to  create  a  positive  environment  for  large  DoD  prime 
contractors  to  voluntarily  provide  assistance  to  third  parties  (eligible  proteges).  During 
the  tenure  of  this  program,  many  large  firms  have  been  the  subject  of  mergers  and  some 
small  firms,  as  they  will,  either  prospered,  w'ere  acquired  by  other  firms,  maintained 
equilibrium  in  the  changing  Defense  marketplace  or  discontinued  operations.  Many 
factors  beyond  the  scope  of  this  program  contribute  to  the  fate  of  these  firms. 
Consequently,  it  is  a  difficult  task  to  isolate  and  assess  the  direct  impact  of  the  MPP  on 
the-  protdge  firms  by  statistical  means. 

However,  it  is  apparent  that  vrithin  the  MPP,  there  have  been  a  significant  number 
of  protege  firms  which  have  witnessed  an  increase  in  Defense-related  technical 
capabilities,  gained  improved  quality  assurance  systems,  received  ISO  certifications,  and 
have  been  awarded  prime  contracts  and  subcontracts  from  DoD  and  other  entities.  GAO 
has  taken  the  position  that  the  survey  conducted  by  the  DoD  did  not  result  in  a  sufficient 
response  rate  (75®/c)  to  permit  GAO  to  make  generalizations  about  the  impact  of  the 
program.  DoD  notes  that  the  response  received  from  the  survey  provides  information  on 
the  positive  impact  of  the  program  upon  many  firms  and,  as  GAO  notes  (see  footnote  12) 
was  sufficient  to  enable  DoD  to  improve  program  administration.  In  fact,  there  are 
various  case  studies  available  on  individual  agreements  which  may  be  useful  to  Congress 
in  determining  the  effectiveness  of  the  program  as  demonstrated  by  this  year's  Nunn- 
Perry  Award  recipients.  (See  Enclosure) 

o 
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See  comment  1 . 


See  comment  2. 


See  comment  3. 


See  comment  1 . 


DoD  concurs  "with  the  recommendation  to  strengthen  performance  review  and 
will  continue  to  make  efforts  to  improve  the  reliability  and  sufficiency  of  data  on  the 
program.  However,  we  note  that  each  individual  agreement  is  unique.  Agreements  are 
defined  by  the  specific  assistance  necessary  to  develop  the  capabilities  of  prot^g^s  within 
very’  diverse  industry  categories.  Consequently,  each  assessment  will  be  unique  and  will 
not  lend  itself  easily  to  a  boilerplate  statistical  analysis.  DoD's  position  is  that  a  case 
study  approach  can  sufficiently  portray  the  impact  of  the  program,  upon  the  protege  firms. 

The  report  notes  that  the  Sendees  have  been  inconsistent  in  their  policies  on 
paying  fees  to  mentors  for  providing  assistance  to  proteges  and  reimbursing  prot^g^s  for 
various  expenses.  The  MPP  is  a  “pilot”  program.  By  allowing  (and  encouraging) 
diversity  of  approaches  the  Depanment  tests  the  range  of  options  for  program  execution 
and  the  impact  of  various  incentives  upon  participation  in  the  program.  In  addition,  the 
nature  of  the  MPP,  as  noted  above,  lends  itself  to  unique  execution  based  on  individual 
agreements  (e.g.  reimbursement  of  prot^g^  expenses  or  payment  of  fees  may  be 
warranted  in  some  cases,  but  not  in  other  instances). 

We  appreciate  GAO's  recognition  of  the  inroads  the  Department  has  made 
recently  in  improving  the  administration  of  the  program.  We  will  continue  in  our 
initiatives  to  enhance  program  management  and  evaluate  program  performance. 

Additional  technical  corrections  were  provided  separately  to  the  GAO  staff. 
Thank  you  again  for  the  opportunity  to  revdew  the  draft  report  and  for  the  opporronity  to 
work  as  a  team  as  this  review  was  conducted. 

2, 

Director,  Office  of  and  Disadvantaged 
Business  Utilization 


Enclosure 
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The  folio-wing  are  gao’s  comments  on  the  Department  of  Defense’s  (dod) 
letter  dated  February  26, 1998. 


GAO  Comments 


1.  DOD  states  that  a  case  study  approach  can  sufficiently  portray  the  impact 
of  the  program  upon  the  protege  firms.  In  this  regard,  dod  provided  nine 
examples  of  mentor-protege  agreements  that  it  believed  illustrated  the 
effectiveness  of  the  program.  However,  there  have  been  close  to  400 
agreements  approved  by  osadbu  since  the  inception  of  the  program.  We  do 
not  beUeve  the  nine  examples,  while  positive  in  themselves,  are  sufficient 
to  assess  the  program’s  overall  effectiveness.  Therefore,  we  have  chosen 
not  to  include  the  enclosure  in  this  report. 

2.  We  have  modified  the  report  to  properly  attribute  to  osadbu  officials  the 
actions  that  they  believe  have  improved  program  administration. 

3.  We  have  incorporated  dod’s  technical  comments  in  the  text  where 
appropriate. 
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Ordering  Information 


The  first  copy  of  each  GAO  report  and  testimony  is  free. 
Additional  copies  are  $2  each.  Orders  should  be  sent  to  the 
following  address,  accompanied  by  a  check  or  money  order 
made  out  to  the  Superintendent  of  Documents,  when 
necessary.  VISA  and  MasterCard  credit  cards  are  accepted,  also. 
Orders  for  100  or  more  copies  to  be  mailed  to  a  single  address 
are  discounted  25  percent. 

Orders  by  mail: 

U.S.  General  Accounting  Office 
P.O.  Box  37050 
Washington,  DC  20013 

or  visit: 

Room  1100 

700  4th  St.  NW  (corner  of  4th  and  G  Sts.  NW) 

U.S.  General  Accounting  Office 
Washington,  DC 

Orders  may  also  be  placed  by  calling  (202)  512-6000 

or  by  using  fax  number  (202)  512-6061,  or  TDD  (202)  512-2537. 

Each  day,  GAO  issues  a  list  of  newly  available  reports  and 
testimony.  To  receive  facsimile  copies  of  the  daily  list  or  any 
list  from  the  past  30  days,  please  call  (202)  512-6000  using  a 
touchtone  phone.  A  recorded  menu  will  provide  information  on 
how  to  obtain  these  lists. 

For  information  on  how  to  access  GAO  reports  on  the  INTERNET, 
send  an  e-mail  message  with  "info"  in  the  body  to: 

info@www.gao.gov 

or  visit  GAO’s  World  Wide  Web  Home  Page  at: 
http://www.gao.gov 
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